{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Event Data Filtering\n",
    "*by: Sebastiaan J. van Zelst*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "Like any data-driven field, the successful application of process mining needs *data munging and crunching*.\n",
    "In pm4py, you can munge and crunch your data in two ways, i.e., you can write ```lambda``` functions and apply them on\n",
    "your event log, or, you can apply pre-built filtering and transformation functions.\n",
    "Hence, in this turtorial, we briefly explain how to filter event data in various different ways in pm4py."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Generic Lambda Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "In a nutshell, a lambda function allows you to specify a function that needs to be applied on a given element.\n",
    "As a simple example, consider the following snippet:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "10"
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f = lambda x: 2 * x\n",
    "f(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "In the code, we assign a ```lambda``` function to variable ```f```.\n",
    "The function specifies that on each possible input it receives, the resulting function that is applied is a multiplication by 2.\n",
    "Hence ```f(1)=2```, ```f(2)=4```, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "Note that, invoking ```f``` only works if we provide an argument that can be combined with the ```* 2``` operation.\n",
    "For example, for ```strings```, the ```* 2``` operation concatenates the input argument with itself:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "'PetePete'"
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f('Pete')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Filter and Map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "Lambda functions allow us to write short, type-independent functions.\n",
    "Given a list of objects, Python provides two core functions that can apply a given lambda function on each element of\n",
    "the given list (in fact, any iterable):"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- ```filter(f,l)```\n",
    " - apply the given lambda function ```f``` as a filter on the iterable ```l```.\n",
    "- ```map(f,l)```\n",
    "  - apply the given lambda function ```f``` as a transformation on the iterable ```l```."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "For more information, study the concept of ‘higher order functions’ in Python, e.g., as introduced [here](https://www.codespeedy.com/higher-order-functions-in-python-map-filter-sorted-reduce/).\n",
    "Let's consider a few simple examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "<filter at 0x1fc59e9a4f0>"
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]\n",
    "filter(lambda n: n >= 5, l)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "The previous example needs little to no explanation, i.e., the filter retains all numbers in the list greater or equal to five.\n",
    "However, what is interesting, is the fact that the resulting objects are not a list (or an iterables), rather a ```filter``` object.\n",
    "Such an objects can be easily transformed to a list by wrapping it with a ```list()``` cast:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "[5, 6, 7, 8, 9, 10]"
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(filter(lambda n: n >= 5, l))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "The same holds for the ```map()``` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "<map at 0x1fc59e7adf0>"
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "map(lambda n: n * 3, l)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "[3, 6, 9, 12, 15, 18, 21, 24, 27, 30]"
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(map(lambda n: n * 3, l))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "Observe that, the previous map function simply muliplies each element of list ```l``` by three."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Lambda-Based Filtering in pm4py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "In pm4py, event log objects mimic lists of traces, which in turn, mimic lists of events.\n",
    "Clearly, ```lambda``` functions can therefore be applied to event logs and traces.\n",
    "However, as we have shown in the previous example, after applying such a lamda-based filter, the resulting object is no longer an event log.\n",
    "Furthermore, casting a filter object or map object to an event log in ```pm4py``` is a bit more involved, i.e., it is\n",
    "not so trivial as ```list(filter(...))``` in the previous example.\n",
    "This is due to the fact that various meta-data is stored in the event log object as well.\n",
    "To this end, pm4py offers wrapper functions that make sure that after applying your higher-order function with a lambda function,\n",
    "the resulting object is again an Event Log object.\n",
    "In the upcoming scripts, we'll take a look at some lambda-based fitlering.\n",
    "First, let's inspect the length of each trace in our running example log by applying a generic map function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "parsing log, completed traces ::   0%|          | 0/6 [00:00<?, ?it/s]",
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "f7c29377364e465f953bc55686df17c3"
      }
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": "[9, 5, 5, 5, 13, 5]"
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pm4py\n",
    "\n",
    "log = pm4py.read_xes('data/running_example.xes', return_legacy_log_object=True)\n",
    "# inspect the length of each trace using a generic map function\n",
    "list(map(lambda t: len(t), log))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "As we can see, there are four traces describing a trace of length 5, one trace of length 9 and one trace of length 13.\n",
    "Let's retain all traces that have a lenght greater than 5."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "[9, 13]"
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lf = pm4py.filter_log(lambda t: len(t) > 5, log)\n",
    "list(map(lambda t: len(t), lf))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "The traces of length 9 and 13 have repeated behavior in them, i.e., the *reinitiate request* activity has been performed at least once:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "[(9, 1), (5, 0), (5, 0), (5, 0), (13, 2), (5, 0)]"
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(map(lambda t: (len(t), len(list(filter(lambda e: e['concept:name'] == 'reinitiate request', t)))), log))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "Observe that the map function maps each trace onto a tuple.\n",
    "The first element describes the length of the trace.\n",
    "The second element describes the number of occurrences of the activity *register request*.\n",
    "Observe that we obtain said counter by filtering the trace, i.e., by retaining only those events that describe the\n",
    "*reinitiate request* activity and counting the length of the resulting list.\n",
    "Note that the traces describe a list of events, and, events are implementing a dictionary.\n",
    "In this case, the activity name is captured by the ```concept:name``` attribute."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "In general, PM4PY supports the following *generic filtering functions*:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- ```pm4py.filter_log(f, log)```\n",
    "  - filter the log according to a function ```f```.\n",
    "- ```pm4py.filter_trace(f,trace)```\n",
    "  - filter the trace according to function ```f```.\n",
    "-  ```pm4py.sort_log(log, key, reverse)```\n",
    "  - sort the event log according to a given ```key```, reversed order if ```reverse==True```.\n",
    "- ```pm4py.sort_trace(trace, key, reverse)```\n",
    "  - sort the trace according to a given ```key```, reversed order if ```reverse==True```."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "Let's see these functions in action:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6\n",
      "2\n"
     ]
    }
   ],
   "source": [
    "print(len(log))\n",
    "lf = pm4py.filter_log(lambda t: len(t) > 5, log)\n",
    "print(len(lf))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9\n",
      "2\n"
     ]
    }
   ],
   "source": [
    "print(len(log[0]))  #log[0] fetches the 1st trace\n",
    "tf = pm4py.filter_trace(lambda e: e['concept:name'] in {'register request', 'pay compensation'}, log[0])\n",
    "print(len(tf))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9\n",
      "5\n",
      "13\n"
     ]
    }
   ],
   "source": [
    "print(len(log[0]))\n",
    "ls = pm4py.sort_log(log, lambda t: len(t))\n",
    "print(len(ls[0]))\n",
    "ls = pm4py.sort_log(log, lambda t: len(t), reverse=True)\n",
    "print(len(ls[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Specific Filters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "There are various pre-built filters in PM4Py, which make commonly needed process mining filtering functionality a lot easier.\n",
    "In the upcoming overview, we briefly give present these functions.\n",
    "We describe how to call them, their main input parameters and their return objects.\n",
    "Note that, all of the filters work on both DataFrames and pm4py event log objects."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Start Activities\n",
    "- ```filter_start_activities(log, activities, retain=True)```\n",
    "  - retains (or drops) the traces that contain the given activity as the final event."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "[{'attributes': {'concept:name': '3'}, 'events': [{'concept:name': 'register request', 'org:resource': 'Pete', 'time:timestamp': datetime.datetime(2010, 12, 30, 14, 32, tzinfo=datetime.timezone(datetime.timedelta(seconds=3600))), 'Activity': 'register request', 'Resource': 'Pete', 'Costs': '50'}, '..', {'concept:name': 'pay compensation', 'org:resource': 'Ellen', 'time:timestamp': datetime.datetime(2011, 1, 15, 10, 45, tzinfo=datetime.timezone(datetime.timedelta(seconds=3600))), 'Activity': 'pay compensation', 'Resource': 'Ellen', 'Costs': '200'}]}, '....', {'attributes': {'concept:name': '4'}, 'events': [{'concept:name': 'register request', 'org:resource': 'Pete', 'time:timestamp': datetime.datetime(2011, 1, 6, 15, 2, tzinfo=datetime.timezone(datetime.timedelta(seconds=3600))), 'Activity': 'register request', 'Resource': 'Pete', 'Costs': '50'}, '..', {'concept:name': 'reject request', 'org:resource': 'Ellen', 'time:timestamp': datetime.datetime(2011, 1, 12, 15, 44, tzinfo=datetime.timezone(datetime.timedelta(seconds=3600))), 'Activity': 'reject request', 'Resource': 'Ellen', 'Costs': '200'}]}]"
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pm4py.filter_start_activities(log, {'register request'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "[]"
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pm4py.filter_start_activities(log, {'register request TYPO!'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "   case:concept:name        concept:name            time:timestamp  costs  \\\n14                 1    register request 2010-12-30 10:02:00+00:00     50   \n15                 1  examine thoroughly 2010-12-31 09:06:00+00:00    400   \n16                 1        check ticket 2011-01-05 14:12:00+00:00    100   \n17                 1              decide 2011-01-06 10:18:00+00:00    200   \n18                 1      reject request 2011-01-07 13:24:00+00:00    200   \n9                  2    register request 2010-12-30 10:32:00+00:00     50   \n10                 2        check ticket 2010-12-30 11:12:00+00:00    100   \n11                 2    examine casually 2010-12-30 13:16:00+00:00    400   \n12                 2              decide 2011-01-05 10:22:00+00:00    200   \n13                 2    pay compensation 2011-01-08 11:05:00+00:00    200   \n0                  3    register request 2010-12-30 13:32:00+00:00     50   \n1                  3    examine casually 2010-12-30 14:06:00+00:00    400   \n2                  3        check ticket 2010-12-30 15:34:00+00:00    100   \n3                  3              decide 2011-01-06 08:18:00+00:00    200   \n4                  3  reinitiate request 2011-01-06 11:18:00+00:00    200   \n5                  3  examine thoroughly 2011-01-06 12:06:00+00:00    400   \n6                  3        check ticket 2011-01-08 10:43:00+00:00    100   \n7                  3              decide 2011-01-09 08:55:00+00:00    200   \n8                  3    pay compensation 2011-01-15 09:45:00+00:00    200   \n37                 4    register request 2011-01-06 14:02:00+00:00     50   \n38                 4        check ticket 2011-01-07 11:06:00+00:00    100   \n39                 4  examine thoroughly 2011-01-08 13:43:00+00:00    400   \n40                 4              decide 2011-01-09 11:02:00+00:00    200   \n41                 4      reject request 2011-01-12 14:44:00+00:00    200   \n24                 5    register request 2011-01-06 08:02:00+00:00     50   \n25                 5    examine casually 2011-01-07 09:16:00+00:00    400   \n26                 5        check ticket 2011-01-08 10:22:00+00:00    100   \n27                 5              decide 2011-01-10 12:28:00+00:00    200   \n28                 5  reinitiate request 2011-01-11 15:18:00+00:00    200   \n29                 5        check ticket 2011-01-14 13:33:00+00:00    100   \n30                 5    examine casually 2011-01-16 14:50:00+00:00    400   \n31                 5              decide 2011-01-19 10:18:00+00:00    200   \n32                 5  reinitiate request 2011-01-20 11:48:00+00:00    200   \n33                 5    examine casually 2011-01-21 08:06:00+00:00    400   \n34                 5        check ticket 2011-01-21 10:34:00+00:00    100   \n35                 5              decide 2011-01-23 12:12:00+00:00    200   \n36                 5      reject request 2011-01-24 13:56:00+00:00    200   \n19                 6    register request 2011-01-06 14:02:00+00:00     50   \n20                 6    examine casually 2011-01-06 15:06:00+00:00    400   \n21                 6        check ticket 2011-01-07 15:22:00+00:00    100   \n22                 6              decide 2011-01-07 15:52:00+00:00    200   \n23                 6    pay compensation 2011-01-16 10:47:00+00:00    200   \n\n   org:resource  @@index  \n14         Pete       14  \n15          Sue       15  \n16         Mike       16  \n17         Sara       17  \n18         Pete       18  \n9          Mike        9  \n10         Mike       10  \n11         Sean       11  \n12         Sara       12  \n13        Ellen       13  \n0          Pete        0  \n1          Mike        1  \n2         Ellen        2  \n3          Sara        3  \n4          Sara        4  \n5          Sean        5  \n6          Pete        6  \n7          Sara        7  \n8         Ellen        8  \n37         Pete       37  \n38         Mike       38  \n39         Sean       39  \n40         Sara       40  \n41        Ellen       41  \n24        Ellen       24  \n25         Mike       25  \n26         Pete       26  \n27         Sara       27  \n28         Sara       28  \n29        Ellen       29  \n30         Mike       30  \n31         Sara       31  \n32         Sara       32  \n33          Sue       33  \n34         Pete       34  \n35         Sara       35  \n36         Mike       36  \n19         Mike       19  \n20        Ellen       20  \n21         Mike       21  \n22         Sara       22  \n23         Mike       23  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>case:concept:name</th>\n      <th>concept:name</th>\n      <th>time:timestamp</th>\n      <th>costs</th>\n      <th>org:resource</th>\n      <th>@@index</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>14</th>\n      <td>1</td>\n      <td>register request</td>\n      <td>2010-12-30 10:02:00+00:00</td>\n      <td>50</td>\n      <td>Pete</td>\n      <td>14</td>\n    </tr>\n    <tr>\n      <th>15</th>\n      <td>1</td>\n      <td>examine thoroughly</td>\n      <td>2010-12-31 09:06:00+00:00</td>\n      <td>400</td>\n      <td>Sue</td>\n      <td>15</td>\n    </tr>\n    <tr>\n      <th>16</th>\n      <td>1</td>\n      <td>check ticket</td>\n      <td>2011-01-05 14:12:00+00:00</td>\n      <td>100</td>\n      <td>Mike</td>\n      <td>16</td>\n    </tr>\n    <tr>\n      <th>17</th>\n      <td>1</td>\n      <td>decide</td>\n      <td>2011-01-06 10:18:00+00:00</td>\n      <td>200</td>\n      <td>Sara</td>\n      <td>17</td>\n    </tr>\n    <tr>\n      <th>18</th>\n      <td>1</td>\n      <td>reject request</td>\n      <td>2011-01-07 13:24:00+00:00</td>\n      <td>200</td>\n      <td>Pete</td>\n      <td>18</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>2</td>\n      <td>register request</td>\n      <td>2010-12-30 10:32:00+00:00</td>\n      <td>50</td>\n      <td>Mike</td>\n      <td>9</td>\n    </tr>\n    <tr>\n      <th>10</th>\n      <td>2</td>\n      <td>check ticket</td>\n      <td>2010-12-30 11:12:00+00:00</td>\n      <td>100</td>\n      <td>Mike</td>\n      <td>10</td>\n    </tr>\n    <tr>\n      <th>11</th>\n      <td>2</td>\n      <td>examine casually</td>\n      <td>2010-12-30 13:16:00+00:00</td>\n      <td>400</td>\n      <td>Sean</td>\n      <td>11</td>\n    </tr>\n    <tr>\n      <th>12</th>\n      <td>2</td>\n      <td>decide</td>\n      <td>2011-01-05 10:22:00+00:00</td>\n      <td>200</td>\n      <td>Sara</td>\n      <td>12</td>\n    </tr>\n    <tr>\n      <th>13</th>\n      <td>2</td>\n      <td>pay compensation</td>\n      <td>2011-01-08 11:05:00+00:00</td>\n      <td>200</td>\n      <td>Ellen</td>\n      <td>13</td>\n    </tr>\n    <tr>\n      <th>0</th>\n      <td>3</td>\n      <td>register request</td>\n      <td>2010-12-30 13:32:00+00:00</td>\n      <td>50</td>\n      <td>Pete</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>3</td>\n      <td>examine casually</td>\n      <td>2010-12-30 14:06:00+00:00</td>\n      <td>400</td>\n      <td>Mike</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>check ticket</td>\n      <td>2010-12-30 15:34:00+00:00</td>\n      <td>100</td>\n      <td>Ellen</td>\n      <td>2</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>3</td>\n      <td>decide</td>\n      <td>2011-01-06 08:18:00+00:00</td>\n      <td>200</td>\n      <td>Sara</td>\n      <td>3</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>3</td>\n      <td>reinitiate request</td>\n      <td>2011-01-06 11:18:00+00:00</td>\n      <td>200</td>\n      <td>Sara</td>\n      <td>4</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>3</td>\n      <td>examine thoroughly</td>\n      <td>2011-01-06 12:06:00+00:00</td>\n      <td>400</td>\n      <td>Sean</td>\n      <td>5</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>3</td>\n      <td>check ticket</td>\n      <td>2011-01-08 10:43:00+00:00</td>\n      <td>100</td>\n      <td>Pete</td>\n      <td>6</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>3</td>\n      <td>decide</td>\n      <td>2011-01-09 08:55:00+00:00</td>\n      <td>200</td>\n      <td>Sara</td>\n      <td>7</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>3</td>\n      <td>pay compensation</td>\n      <td>2011-01-15 09:45:00+00:00</td>\n      <td>200</td>\n      <td>Ellen</td>\n      <td>8</td>\n    </tr>\n    <tr>\n      <th>37</th>\n      <td>4</td>\n      <td>register request</td>\n      <td>2011-01-06 14:02:00+00:00</td>\n      <td>50</td>\n      <td>Pete</td>\n      <td>37</td>\n    </tr>\n    <tr>\n      <th>38</th>\n      <td>4</td>\n      <td>check ticket</td>\n      <td>2011-01-07 11:06:00+00:00</td>\n      <td>100</td>\n      <td>Mike</td>\n      <td>38</td>\n    </tr>\n    <tr>\n      <th>39</th>\n      <td>4</td>\n      <td>examine thoroughly</td>\n      <td>2011-01-08 13:43:00+00:00</td>\n      <td>400</td>\n      <td>Sean</td>\n      <td>39</td>\n    </tr>\n    <tr>\n      <th>40</th>\n      <td>4</td>\n      <td>decide</td>\n      <td>2011-01-09 11:02:00+00:00</td>\n      <td>200</td>\n      <td>Sara</td>\n      <td>40</td>\n    </tr>\n    <tr>\n      <th>41</th>\n      <td>4</td>\n      <td>reject request</td>\n      <td>2011-01-12 14:44:00+00:00</td>\n      <td>200</td>\n      <td>Ellen</td>\n      <td>41</td>\n    </tr>\n    <tr>\n      <th>24</th>\n      <td>5</td>\n      <td>register request</td>\n      <td>2011-01-06 08:02:00+00:00</td>\n      <td>50</td>\n      <td>Ellen</td>\n      <td>24</td>\n    </tr>\n    <tr>\n      <th>25</th>\n      <td>5</td>\n      <td>examine casually</td>\n      <td>2011-01-07 09:16:00+00:00</td>\n      <td>400</td>\n      <td>Mike</td>\n      <td>25</td>\n    </tr>\n    <tr>\n      <th>26</th>\n      <td>5</td>\n      <td>check ticket</td>\n      <td>2011-01-08 10:22:00+00:00</td>\n      <td>100</td>\n      <td>Pete</td>\n      <td>26</td>\n    </tr>\n    <tr>\n      <th>27</th>\n      <td>5</td>\n      <td>decide</td>\n      <td>2011-01-10 12:28:00+00:00</td>\n      <td>200</td>\n      <td>Sara</td>\n      <td>27</td>\n    </tr>\n    <tr>\n      <th>28</th>\n      <td>5</td>\n      <td>reinitiate request</td>\n      <td>2011-01-11 15:18:00+00:00</td>\n      <td>200</td>\n      <td>Sara</td>\n      <td>28</td>\n    </tr>\n    <tr>\n      <th>29</th>\n      <td>5</td>\n      <td>check ticket</td>\n      <td>2011-01-14 13:33:00+00:00</td>\n      <td>100</td>\n      <td>Ellen</td>\n      <td>29</td>\n    </tr>\n    <tr>\n      <th>30</th>\n      <td>5</td>\n      <td>examine casually</td>\n      <td>2011-01-16 14:50:00+00:00</td>\n      <td>400</td>\n      <td>Mike</td>\n      <td>30</td>\n    </tr>\n    <tr>\n      <th>31</th>\n      <td>5</td>\n      <td>decide</td>\n      <td>2011-01-19 10:18:00+00:00</td>\n      <td>200</td>\n      <td>Sara</td>\n      <td>31</td>\n    </tr>\n    <tr>\n      <th>32</th>\n      <td>5</td>\n      <td>reinitiate request</td>\n      <td>2011-01-20 11:48:00+00:00</td>\n      <td>200</td>\n      <td>Sara</td>\n      <td>32</td>\n    </tr>\n    <tr>\n      <th>33</th>\n      <td>5</td>\n      <td>examine casually</td>\n      <td>2011-01-21 08:06:00+00:00</td>\n      <td>400</td>\n      <td>Sue</td>\n      <td>33</td>\n    </tr>\n    <tr>\n      <th>34</th>\n      <td>5</td>\n      <td>check ticket</td>\n      <td>2011-01-21 10:34:00+00:00</td>\n      <td>100</td>\n      <td>Pete</td>\n      <td>34</td>\n    </tr>\n    <tr>\n      <th>35</th>\n      <td>5</td>\n      <td>decide</td>\n      <td>2011-01-23 12:12:00+00:00</td>\n      <td>200</td>\n      <td>Sara</td>\n      <td>35</td>\n    </tr>\n    <tr>\n      <th>36</th>\n      <td>5</td>\n      <td>reject request</td>\n      <td>2011-01-24 13:56:00+00:00</td>\n      <td>200</td>\n      <td>Mike</td>\n      <td>36</td>\n    </tr>\n    <tr>\n      <th>19</th>\n      <td>6</td>\n      <td>register request</td>\n      <td>2011-01-06 14:02:00+00:00</td>\n      <td>50</td>\n      <td>Mike</td>\n      <td>19</td>\n    </tr>\n    <tr>\n      <th>20</th>\n      <td>6</td>\n      <td>examine casually</td>\n      <td>2011-01-06 15:06:00+00:00</td>\n      <td>400</td>\n      <td>Ellen</td>\n      <td>20</td>\n    </tr>\n    <tr>\n      <th>21</th>\n      <td>6</td>\n      <td>check ticket</td>\n      <td>2011-01-07 15:22:00+00:00</td>\n      <td>100</td>\n      <td>Mike</td>\n      <td>21</td>\n    </tr>\n    <tr>\n      <th>22</th>\n      <td>6</td>\n      <td>decide</td>\n      <td>2011-01-07 15:52:00+00:00</td>\n      <td>200</td>\n      <td>Sara</td>\n      <td>22</td>\n    </tr>\n    <tr>\n      <th>23</th>\n      <td>6</td>\n      <td>pay compensation</td>\n      <td>2011-01-16 10:47:00+00:00</td>\n      <td>200</td>\n      <td>Mike</td>\n      <td>23</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas\n",
    "\n",
    "ldf = pm4py.format_dataframe(pandas.read_csv('data/running_example.csv', sep=';'), case_id='case_id',\n",
    "                             activity_key='activity', timestamp_key='timestamp')\n",
    "pm4py.filter_start_activities(ldf, {'register request'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "Empty DataFrame\nColumns: [case:concept:name, concept:name, time:timestamp, costs, org:resource, @@index]\nIndex: []",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>case:concept:name</th>\n      <th>concept:name</th>\n      <th>time:timestamp</th>\n      <th>costs</th>\n      <th>org:resource</th>\n      <th>@@index</th>\n    </tr>\n  </thead>\n  <tbody>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pm4py.filter_start_activities(ldf, {'register request TYPO!'})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### End Activities\n",
    "- ```filter_end_activities(log, activities, retain=True)```\n",
    "  - retains (or drops) the traces that contain the given activity as the final event."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "For example, we can retain the number of cases that end with a \"payment of the compensation\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "3"
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(pm4py.filter_end_activities(log, 'pay compensation'))"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Event Attribute Values"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "- ```filter_event_attribute_values(log, attribute_key, values, level=\"case\", retain=True)```\n",
    "  - retains (or drops) traces (or events) based on a given collection of ```values``` that need to be matched for the\n",
    "  given ```attribute_key```. If ```level=='case'```, complete traces are matched (or dropped if ```retain==False```) that\n",
    "  have at least one event that describes a specifeid value for the given attribute. If ```level=='event'```, only events\n",
    "  that match are retained (or dropped)."
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "outputs": [
    {
     "data": {
      "text/plain": "[['Pete', 'Mike', 'Ellen', 'Sara', 'Sara', 'Sean', 'Pete', 'Sara', 'Ellen'],\n ['Mike', 'Mike', 'Sean', 'Sara', 'Ellen'],\n ['Pete', 'Sue', 'Mike', 'Sara', 'Pete'],\n ['Mike', 'Ellen', 'Mike', 'Sara', 'Mike'],\n ['Ellen',\n  'Mike',\n  'Pete',\n  'Sara',\n  'Sara',\n  'Ellen',\n  'Mike',\n  'Sara',\n  'Sara',\n  'Sue',\n  'Pete',\n  'Sara',\n  'Mike'],\n ['Pete', 'Mike', 'Sean', 'Sara', 'Ellen']]"
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# retain any case that has either Peter or Mike working on it\n",
    "lf = pm4py.filter_event_attribute_values(log, 'org:resource', {'Pete', 'Mike'})\n",
    "list(map(lambda t: list(map(lambda e: e['org:resource'], t)), lf))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "outputs": [
    {
     "data": {
      "text/plain": "[['Pete', 'Mike', 'Pete'],\n ['Mike', 'Mike'],\n ['Pete', 'Mike', 'Pete'],\n ['Mike', 'Mike', 'Mike'],\n ['Mike', 'Pete', 'Mike', 'Pete', 'Mike'],\n ['Pete', 'Mike']]"
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# retain only those events that have Pete or Mik working on it\n",
    "lf = pm4py.filter_event_attribute_values(log, 'org:resource', {'Pete', 'Mike'}, level='event')\n",
    "list(map(lambda t: list(map(lambda e: e['org:resource'], t)), lf))\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "rise": {
   "enable_chalkboard": true,
   "footer": "<h1>footer<h1>",
   "overlay": "<div><h1>pm4py</h1></div><div><h2>Event Data Filtering</h2></div>"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}